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Abstract. We show that Complementary Series representations 
of SO{n, 1), which are sufficiently close to a cohomological repre- 
sentation contain discretely, Complementary Series of SO{m, 1) 
also sufficiently close to cohomological representations, provided 
that the degree of the cohomological representation does not ex- 
ceed m/2. 

We prove, as a consequence, that the cohomological represen- 
tation of degree i of the group SO{ti, 1) contains discretely, the 
cohomological representation of degree i of the subgroup SO{m, 1) 
if i < m/2. 

As a global application, we show that if G/Q is a semisimple 
algebraic group such that G(R) — SO{n,l) up to compact fac- 
tors, and if we assume that for all n, the tempered cohomologi- 
cal representations are not limits of complementary series in the 
automorphic dual of SO{n, 1), then for all n, non-tempered coho- 
mological representations are isolated in the automorphic dual of 
G. This reduces conjectures of Bergeron to the case of tempered 
cohomological representations. 
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1. Introduction 

A famous Theorem of Selberg jSelj says, that the non-zero eigen- 
values of the Laplacian acting on functions on quotients of the upper 
half plane by congruence subgroups of the integral modular group are 
bounded away from zero, as the congruence subgroup varies. In fact, 
Selberg proved that every non-zero eigenvalue A of the Laplacian on 
functions on r\f), F C 5'L2(Z), satisfies the inequality 



A Theorem of Clozel [Cloj generalises this result to any congreunce 
quotient of any symmetric space of non-compact type: if G is a linear 
semi-simple group defined over Q, F C G(Z) a congruence subgroup 
and X = G{M.)/K the symmetric space of G, then nonzero eigenvalues 
A of the Laplacian acting on the space of functions on T\X satisfy: 

A > e, 

where e > is a number independent of the congruence subgroup F. 

Analogous questions on Laplacians acting on differential forms of 
higher degree (functions may be thought of as differential forms of 
degree zero) have geometric implications on the cohomology of the 
locally symmetric space. Concerning the eigenvalues of the Laplacian, 
Bergeron ( [Ber] ) has conjectured the following: 

Conjecture 1. (Bergeron) Let X be the real hyperbolic ra-space and 
F C SO{n, 1) a congruence arithmetic subgroup. The non-zero eigen- 
values A of the Laplacian acting on the space n*(F\X) of differential 
forms of degree i satisfy: 

A > e, 

for some e > independent of the congruence subgroup F, provided 
i is strictly less than the "middle dimension" (i.e. i < [n/2]). 

If n is even, he conjectures that the above conclusion holds even if 
i = [n/2] = n/2. (When n is odd, there is a slightly more technical 
statement which we omit). 

In this paper, we show, for example, that if the above conjecture 
holds true in the middle degree for all even integers n, then the conjec- 
ture holds for arbitrary degrees of the differential forms (See Theorem 
|3]). For odd n, there is, again, a more technical statement (see Theorem 
U part (2)). 
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The main Theorem of the present paper is Theorem [1] on the oc- 
currence of some discrete components of certain Complementary Se- 
ries Representations associated to SO{n, 1). The statement on Lapla- 
cians may be deduced from the main theorem, using the Burger-Sarnak 
( |Bu-Sa] ) method. 

We now describe the main theorem more precisely. Let i < [n/2] — l 
and G = SO{n, 1). Let P = MAN be the Langlands decomposition of 
P, K G G a maximal compact subgroup of G containing M. Then K 
is isomorphic to the orthogonal group 0{n) and M = 0{n — l); denote 
by p the standard representation of 0(n), tensored by the determinant 
character; if g, t denote the complexified Lie algebras of G and K, then 
the Cartan decomposition takes the form g = t © p, as i^'-modules. 

Let, analogously, pM be the standard representation of M twisted 
by the character such that this representation occurs in the restriction 
of p to the subgroup M of K. Let A* be its i-th exterior power repre- 
sentation. Denote by pp the character of P acting on the top exterior 
power of the Lie algebra of A^. Consider the induced representation 
(with normalized induction) 

= Ind${A' ppia)"") 

for < M < 1 — The representation Tru{i) denotes the completion 
of the space TTu{i) of i^"-finite functions with respect to the G-invariant 
metric on 7r„(z), and is called the complementary series representa- 
tion corresponding to the representation A' of M and the parameter u. 

Let H = SO{n — 1,1) be embedded in G such that P fl if is a 
maximal parabolic subgroup of if, A C H, and K (1 H a maximal 
compact subgroup of H. We now assume that < u < 1 — Let 

/ _ (n-i)u-i ^Yien < u' < 1 Denote by AV the i-th exterior 

power of the standard representation of M fl if ^ 0(?2 — 2). 

We obtain analogously the complementary series representation 
of H. The main theorem of the paper is the following. 



Theorem 1. If 

1 . 2i 
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then the complementary series representation cr„/(i) occurs discretely 

in the restriction of the complementary series representation vr„(z) of 
G = SO{n, 1), to the subgroup H = SO{n — 1, 1); 

C T^u{.i)\so{n-i,iy 

Remark 1. The corresponding statement is false for the space of K- 
finite vectors in both the spaces; this inclusion holds only at the level 
of completions of the representations involved. 

As u tends to the limit 1 — the representations 7r„(i) tend (in 

the Fell topology on the unitary dual G) both to the representation 

Ai = Ai{n) and to Aj+i = Aj+i(n); we denote by Aj{n) the unique 
cohomological representation of G which has cohomology (with trivial 
coefficients) in degree j and by Aj{n) the space of i^-finite vectors in 

Aj{n). Using this, and the proof of Theorem [H we obtain 

Theorem 2. The restriction of the cohomological representation Ai{n) 
of SO{n, 1) to the subgroup H = SO{n — 1, 1) contains discretely, the 

cohomological representation Ai{n — 1) of SO{n — 1, l); 

Ai{n-1) C Ai(r^)|5o(n-l,l)• 

Supose now that G is a semi-simple linear algebraic group defined 
over Q and Q-simple, such that 

G{R) ~ SO{n,l) 

up to compact factors with ri > 4 (if n = 7, we assume in addition that 
G is not the inner form of some trialitarian D4). Then there exists a 
Q-simple Q-subgroup Hi G G such that 

Hi{R) ~ SO{n- 2,1), 

up to compact factors. 

Denote by Gaui the "automorphic dual" of SO{n, 1), in the sense of 
Burger-Sarnak ( |Bu-Sa] ). Suppose Ai = Ai{n) is a limit of representa- 
tions pm in GAut- The structure of the unitary dual of SO{n, 1) shows 

that this means that = vr„^(i) for some sequence which tends 

from the left, to 1 ^ (or to 1 — we will concentrate only on 

the first case, for ease of exposition). 

Since = 7^um(*) ^ GAut, a result of Burger-Sarnak ( |Bu-Saj ). 
implies that 

res(p^) C HAut- 
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Applying Theorem [T] twice, we get 

5m = ^u'^ii) e res(p^) C HAut 

Taking limits as m tends to infinity, we get Ai{n — 2) as a limit of 

representations Om in HAut- Therefore, the isolation of Ai{n) in GAut 
is reduced to that for SO{n — 2, 1), and so on. We can finally assume 

that Ai{m) is a tempered representation of SO{m, 1) where m = 2i or 
2i + 1. 

This proves the following Theorem. 

Theorem 3. If for all m, the tempered cohomological representation 
Ai{m) (i.e. when i = [m/2]) is not a limit of complementary series 
in the automorphic dual of SO{m, 1), then for all integers n, and for 

i < [1^/2], the cohomological representation Ai{n) is isolated in the 
automorphic dual of SO{n, 1). 

The major part of the paper is devoted to proving the main theorem 
(Theorem [T]). The proof of Theorem [T] proceeds as follows. 

(1) We first prove Theorem [1] when z = 0; that is tIu = ^m(O) is an 
unramified representation and < m < 1. In this case, we get a 
model of the representation tt^ by restricting the functions (sections of 
a line bundle) on G/P to the big Bruhat cell Nw ~ M"~^ and taking 
their Fourier transforms. The G-invariant metric is particularly easy to 
work with on the Fourier transforms (see Theorem [T0|) ; it is then easy 
to see that there is an isometric embedding J : a„/ — > (Theorem [T5|) . 

(2) We now describe the proof of Theorem [T] in the ramified case. 
The representation 7iu{i) is the "completion" of the space of Schwartz 
class functions on the big Bruhat cell, with values in the representa- 
tion A* of the compact group M, with respect to a bilinear form. That 
this bilinear form is an inner product is not so clear (in contrast to 
the unramified case); we assume this ( [Ba-Bj : in [Ba-B] . they prove the 
existence of a positive definite inner product on the space of i^-finite 

vectors in 7ru{i)', since our bilinear form is also G-invariant, the irre- 
ducibility of the module involved shows that these two bilinear forms 
coincide. 

We again get a model of the representations TTu{i) on the space of 
Fourier transforms of Schwartz class functions on the big Bruhat Cell 
in G/P. There is an embedding J : au'{i) — > 7r„(z), which can be 
explicitly described. We prove that the adjoint J* of J is essentially the 
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restriction 7r_u(i) a"_„(i). This proves that the map J is equivariant 
for the action of the subgroup H and estabhshes the main theorem 
(Theorem [1]) for arbitrary i. 
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2. Preliminary Results and Notation 

2.1. The Group G = SO{n,l) and the representations 7ru{i)- 
The group G — SO{n, 1) is the subgroup of SLn+i{M.) preserving the 
quadratic form 

XqXyi ~t~ X-^ ~t~ X2 ~t~ * * * ~t~ X^^-^j 

on R""*"-^. Denote by eo.ei,-- - ,e„_i,e„ the standard basis of M""*"^. 
Denote by w the matrix in iS'L„+i(]R), which takes the basis element 
Co to e„, e„ to — Cq and takes the other basis elements into for 
1 < i < n — 1. Then w has the matrix form 

0„_i 1' 
*0 
-1 0„_i 0^ 

where 0„_i is the zero vector in R"~^ viewed as a row vector of size 
n — 1 and *0 refers to its transpose (a column vector of size n — 1 all 
of whose entries are zero) and refers to the identity matrix of size 
n — 1. The group G is the matrix group 

{g e GLn+i{R) :* wg = tv}. 

Denote by P the subgroup of G which takes the basis vector cq into 
a multiple of itself. Let A be the group of diagonal matrices in G, M 
the maximal compact subgroup of the centralizer of A in G. Let N be 
the unipotent radical of P. Then, 



a 


On- 


1 





*0 


In- 


1 


*0 





On- 


1 


a-' 



A = {d{a) = *0 In-i *0 : a e R*}, 
Similarly M consists of matrices of the form 

: m e SO{n-l)}. 




'n-l 





m 




'n-l 


±1 



a 







X 








n-l 




'n-l 


1 



: X e R"-'}, 



where x G R"^^ is thought of as a row vector. Then P = MAN is the 
Langlands decomposition of P. The map x 1— > u{x) from R"~^ into 
is an isomorphism of algebraic groups. 
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Let K — G r\0{n + 1). Then is a maximal compact subgroup 
of G and is isomorphic to 0{n) x Let pp{a) be the character 

whose square is the determinant of Ad[d{a)) acting on the complex Lie 
algebra n = Lie{N) C. Then pp{a) — a~ . 

Write = Lie{G) O C, t = Lie{K) O C and set = e p, the 
Cartan Decomposition which is complexified. As a representation of 
K — 0(n), the space p = C" is the standard representation twisted 
by a character. Denote by pM = C"^^ the standard representation 
of M = 0(n — 1) twisted by that character so that pM occurs in p 
restricted to K. Let i be an integer with 0<i<[n/2] — 1 where [ ] 
denotes the integral part. Denote by A^pM the representation of M on 
the i-th exterior power of pM- 

UueC and p = man e MAN = P, the map 

P ^ A*pAf(m) (g) pp(a)" (g) trivN{n) = A*(m)pp(a)" 
is a representation of P. Define 

7Tu{i) = Ind${A^pM ® Pp{aT ® liv) 

to be the space of functions on G with values in the vector space A*pM 
which satisfy 

figman) = A\m)ifig))ppia)-^'+-\ 

for all {g,m,a,n) E G X M X A X xN. In addition, the function / is 
assumed to be i^-finite. The space 7r„(i) is a (g, i^)-module. The rep- 
resentation TTuii) is called the complementary series of SO{n, 1) with 
parameter u and A*. 

When i = 0, the representation 7r„(0) has a i^-fixed vector, namely 
the function which satisfies f{kan) = pp(a)~(^+") for all kan e KAN — 
G. We denote 7r„(0) = 7r„; we refer to this representation as the un- 
ramified complementary series of SO{n, 1) with parameter u. 

2.2. Bruhat Decomposition of SO{n, 1). We have the Bruhat De- 
composition of 0{n, 1): 

(1) G = NwMAN U MAN. 

That is, every matrix g E G = 0{n,l) which does not lie in the para- 
bolic subgroup P, may be written in the form g = u{x)wmd{a')u{y) for 
some x,y & M"~^, a' e IR* and m e M or in the form g = md{a)u{y). 
In these coordinates, the set NwP — NwMAN is an open subset of 
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G of full Haar measure. The map x ^— u{x) is an isomorphism of 
groups from M"^^ onto A^. Let dn denote the image of the Lebesgue 
measure on M"^^ under this map. Denote by d*a the measure for 
a' G M* ^ A via the map a' ^ d{a'). Under the Bruhat decomposition 
(equation ([1]), there exists a choice of Haar measure on M such that 

(2) dg = dx d*a' dm dy pp[a'Y. 

Given x = {xi, X2, - ■ ■ , Xk) G M''", denote by | a; p the sum 

k 

I |2 \^ 2 

Lemma 4. If x E M"^^ and x 7^ 0, then the Bruhat decomposition of 
the element g = wu{x)w is given by 

g = u{z)wm{x)d{a')u{y) 

with 

z = - — --, m = l„_i - 2- — --, a = — — and y = x 
I X 1"^ I X 1^ 2 

(note that *xx is a square matrix of size n—1 and that m is the reflection 
on the orthogonal complement of x). 

The proof is by multiplying out the right hand side and explicitly 
computing the matrix product. 

We will find it convenient to compute the Jacobian of the transfor- 
mation X 1-^ 1^ for X e ]R"~^ \ {0}. 

Lemma 5. /// G Cc(]R"'^"'^\{0}) is a continuous function with compact 
support, then we have the formula 

[ dxf{-^) = [ -r^^— /(x). 

Proof. In polar coordinates, x = rp with r > and p in the unit sphere 
gn-2 q£ Then the Lebesgue measure dx has the form dx = 

r"~^^dfi{p) where /i is a rotation invariant measure on S*""^. In polar 
coordinates, the transformation x ^ takes the form {r,p) i-^ {j.,p)- 

Therefore d{j^) = ^^d^ = □ 
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2.3. Iwasawa Decomposition of 0(n, 1). We have the Iwasawa de- 
composition of 0{n, 1) = G: 

(3) G = KAN. 

That is, every matrix g in SO{n, 1) may be written as a product g = 
kd{a)u{y) with k eK^aeW and y G W'^. 

Lemma 6. The Iwasawa decomposition of the matrix u{x)w is of the 
form 

u{x)w = k{x)d{a)u{y), 

I |2 

with a = 1 + ^ and y = f = — ^jr- The matrix k{x) is given by the 
formula 

k{x) = u{x)wu{—y)d{a~^) . 

Proof. Put g — u{x)w — kd{a)u{y). Compute the product 

w^u{x)u{x)w gg u{y)d{a)d{a)u{y) . 

By comparing the matrix entries on both sides of this equation, we 
get the Lemma. □ 

We normalize the Haar measures dgf, dk d*a and dy on G, K, A, N 
respectively so that 

(4) dg = pp{aY dk d*a dy. 

{dy is the Lebesgue measure on ^ ]R"~^, on A ~ M* the measure is 

\a\ 



d*a = ^^4p where dia is the Lebesgue measure). 



2.4. The space and a linecir form on J^i. Define J^i to be 
the space of complex valued continuous functions f on G such for all 
{g,m,a,n) e G X M X A X N we have 

figman) = pp{a)~^f{g). 

The group G acts on J^i by left translations. Denote by Cc{G) the 
space of complex valued continuous functions with compact support. 
The following result is well known. 

Lemma 7. The map 

f ^ f{9) \ f{gfnan)pp{a)^ dm dn d*a, 

J MAN 

is a surjection from Cc{G) onto the space T\. 
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It follows from this and the Iwasawa decomposition (see equation 
dl])), that the map 

(5) L(f) = [ f{k)dk 

J K 

is a G-invariant linear form on the space J^i. 

Lemma 8. For every function ip & Ti we have the formula 

L{4>) = / dx(j){u{x)w) = I dk(f){k). 

Proof. Given / G Cc{G), we have from equation ([2]) that 

/ dxf{u{x)w) = / f{g)dg = L(J). 

On the other hand, by equation (jlj), we have 

/ dkj{k)= [ f{g)dg = L(f). 
Jk Jg 

These two equations show that the lemma holds if = /. 

It follows from Lemma [7] that every function G is of the form 
/ for some /. This proves the Lemma. 

□ 



12 B.SPEH AND T. N. VENKATARAMANA 

3. Unramified Complementary Series for SO{n, 1) 

3.1. A Model for Unramified Complementary Series. Let < 

?x < 1, and ly = W'-^. Denote by S{W) the Schwartz Space of W. 

Let K = 0{n) and M = 0(n — 1). Then K/M is the unit sphere in 
W . Denote by w the non-trivial Weyl group element of SO{n, 1). The 
space G/P is homogeneous for the action of K and the isotropy at P is 
M. Therefore, G/P = K/M. The Bruhat decomposition implies that 
we have an embedding C K/M via the map n i— > nwP G G/P = 
K/M. Denote by J-'u the space of continuous functions on G which 
transform according to the formula 

figman) = f{g)pp{ap'+-\ 

for all G G, m G M, a G A and n G N. An element of the Schwartz 
space S(W) on FT ~ A may be extended to a continuous function on 
G, which lies in the space 

We will view S{W) as a subspace of JF„ via this identification. 

Lemma 9. If < u < 1 and G S{W) C J-'u, the integral 

Iu{4>){9) = / dx(f){gu{x)w) 
Jw 

converges. 

The integral I{u){(f)) lies in the space T^u- 

Proof. The Iwasawa decomposition of u{x)w (Lemma [6]) implies that 
u{x)w = kman with 



1 + Ni)(n-l)(l+«)/2 



where | x p is the standard inner product of n = ■u(a;) G A ~ in the 
space W and k G K, m G M and n E N. Therefore, for large x G W, 
pp(a)^^" ~| X which proves the convergence of the integral. 

The last statement is routine to check. □ 

There is a G-invariant pairing between JF^ and defined by 



<(^, ?/'>= / dk(j){k)ip{k), 



K 



for all ip G J-'u and ip G J-'-u- Here denotes the complex conjugate 
of 0(A;). 
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Observe that the product 4>{g)ip{g) of the elements (p E J^u and 
ip G J^-u hes in the space J^i. Therefore, < 0, >= L{(f)ip) where L is 
the invariant hnear form on J-'i as in equation ([5]). 

From Lemma E] it follows, for (p E J^u and ip ^ ^-u, that 



Given (p G S{W), denote by (p its Fourier transform. 

Theorem 10. For < u < I and cp G S{W) C Tu, we have the 
formula 



relating the pairing between Tu and T-u, and a norm on the Schwartz 
space, (here c is a constant depending only on the constant u). 

In particular, the pairing < (p,I{u){(p) > is positive definite on the 
Schwartz space S(W). 

The proof of Theorem [TO] will occupy the rest of the section (3.1). 
Note that Theorem [10] is already proved in the paper of Grave- Viskik 
( [G V] ) . The proof we give below is a little different, and moreover, we 
need the functional equation that is proved below, in the course of the 
proof of Theorem [TO] 

Lemma 11. If (p* G S{W) C Tu, then we have the formula 



where n,n' E N correspond respectively to y,x E W . 

Proof. We view S(W) as a subspace of JF^. Thus, the function </)*(x) G 
S{W) is identified with the section x i— > (p*{u{x)w) with (p* G J-'u- We 
then get the equality of the integrals: 







Using the fact that (p* lies in JF^, we see that 



<P*{u{y)wu{x)w) = <P*{u{y + 
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Therefore, the integral of the lemma is 



|2 



W 



X 



2x 



We now use the computation of the Jacobian of the map x 
Lemma [5]) to get 

/ c^\nwn'w)dn' = [ + 
Jn Jw U ^ I /^J ^ 

and this is easily seen to be the right hand side of the equation of the 

Lemma. □ 

Recall that 

I (u) {(j)*) (nw) = / dn (f)* {nwn w) . 
Jn 

From Lemma [8] we have 

<0*,/(u)(0*) >= [ dn{(P*{nw)I{u){(l)*){nw)). 
Jn 

Therefore, it follows from the preceding Lemma that the G-invariant 
pairing < (j)* , I{u){(j)*) > is the integral 



w Jw 



The Fubini Theorem implies that this integral is 

1 



dx , ^ / dy(j){y)(l){y + x). 



w I I Jw 

2 



The inner integral is the L inner product between and its translate by 
X. Since the Fourier transform preserves the inner product and converts 
translation by x into multiplication by the character y ^-^ ^-^ix.y^ 
follows that the integral becomes 

d^ .^^ul.n^,. [ dy I m ? e-'--y. 



I rf |(n— 1)(1 — U) I 

W \X \^ ' Jw 

The integral over y is simply the Fourier transform. Therefore, we have 
(6) <0,/(n)(0)>= /" c/x , I fe) 



w 



X 



Lemma 12. Let f G S{W), and s a complex number with real part 
positive and less than {n — l)/2. Then we have the functional equation 

r{s) [ dx-\-J{x)=T{^-s) f dx\xr'-'^ f{x), 
Jw 1^1 ^ JW 

where T is the classical Gamma function. 
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Proof. If s is a complex number with positive real part, then T{s) is 
defined by the integral 

Denote by L{s, /) the integral 




Multiplying L{2s, /) by r(s), and using Fubini, we obtain 

r(.)L(2s,/)= / re-H^^. 
Jw 1^1 -'0 

Changing the variable t to \ x \^ t and using Fubini again, we get 

Tis)Li2sJ)= / -f / e-'\^\7ix)dx. 
Jo ^ Jw 

The inner integral is the inner product in L'^(W) of the functions e~*'^'^ 
and /. The Fourier transform preserves this inner product. Therefore, 
the inner product is the inner product of their Fourier transforms. 

The Fourier transform of e~*'^'^ is ^(„}i)/2 G~^^^^^^- The Fourier trans- 
form of / is /(— x). Therefore, we have the equality 

T{s)L{2sJ)= r^t-(-i)/2 f e-^^^"/'f{-x)dx. 
Jo t Jw 

In this double integral, change t to | x p /t. We then get 

T{s)L{2s, 7) = r ^t("-i)/2--e-* / I X l^^-^^-^) f{x)dx. 
Jo t Jw 

This proves the functional equation. 

□ 

Theorem [10] is now an immediate consequence of equation ([6]) and 
of the functional equation in Lemma [12] (applied to the value s = 

Notation 1. We denote by tt^ the completion of the space S{W) 
under the metric defined by Theorem [TO] We then get a metric on 
TT^. The G-invariance of the pairing between JF„ and and the 
definition of the inner product in Theorem [10] implies that the metric 
on Tiu is G-invariant. Note that elements of the completion niay 
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not be measurable functions on G, but only "generalised functions" or 
distributions. 

3.2. Embedding of the unramified Complementary series. 

Notation 2. Consider H = 0{n — 1,1) C 0(n, 1) = G embedded by 
fixing the n — 1-th basis vector e„_i. Denote by the representation 
TTy/ constructed in the last section, not for the group G, but, for the 
group H. We set u' — ^""^^2"^ • We assume that < u' < 1 which 
means that ^-r < u < 1. 

n—l 

Denote by W the space M""^ Let H^' C be the subspace M""^ 
whose last co-ordinate in ]R"~^ is zero. 

Let Wu be the completion of the Schwarz space of W under the 
metric 

The convergence of this integral near infinity is clear since (j) lies in 
the Schwartz space; the convergence of the integral near follows 
by converting to polar co-ordinates and by using the assumption that 
0<u<l. 

Define the space W'u' correspondingly for W' as the completion of 
the Schwartz space of W under the metric 

Now, Wu consists of all measurable functions (p onW = M"~^ which 
the li-norm, that is, integral 

is finite. One may define similarly, the it'-norm of a function on W'. 
Given in the space S{W), we define the function J((/>) on W by 
setting, for all {y, t) eW xR^ K'^-^ x K ~ K'^-^ = W, 

J{ct>{y,t))^m- 
Then J {(f)) is a bounded measurable function on R"""^. 

Proposition 13. The map J defined on the Schwartz space S{W') 
has its image in the Hilbert space Wu and extends to an isometry from 

W'u' ^ Wu. 
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Proof. We compute the w-norm of the bounded function J(0): 

JWxR (I ?/ p -1-^2^^— 

By changing the variable t to \ y \ t and using the definition of the map 
J, we see that this w-norm is equal to the integral 



/I , (n— l)ti / I |(n-l)u-l 

(1 + 1^)^— jvK' I y r ^ 

The integral over t converges when the exponent of the denominator 
term is strictly larger than 1; that is, u > The integral over 

y is simply the u' norm of since (n— — 1 = {n—2)u'. Therefore, the 
M-norm of J{4>) is the w'-norm of up to a constant (the integral over 
the variable t of the function ^^^^2^^„^i)^ ) ■ We have therefore proved 
the proposition. □ 

The completion 7f„ (and similarly au>) was defined with respect to 
the metric < 0, 1{u){(f)) > (we use Theorem[Tn]to say that this is indeed 
a metric). It is clear from this definition that for G S(W) C J-'u, the 
map (f) ^ (f) (the roof over refers to the Fourier transform) gives an 
isometry from onto Wu- Similarly du' is isometric to W'u'- There- 
fore, Proposition [13] says that the Hilbert space du' is isometrically 
embedded in the Hilbert space 7r„. We now show that the map J is 
equivariant with respect to the action of the subgroup H on both sides. 

Notation 3. Let 0* G S(W) C JF^ be a function whose Fourier trans- 
form is a smooth compactly supported function 0* on M"~^ \ {0}. De- 
note by G J^-u the image of 0* under the map I{u). By the definition 
of I{u), we have the formula (see Lemma [TTl) . 



(f){u{y)w) = / (j)*{u{y)wu{x)w)dx. 
J 

Using Lemma [121 we now get 



^1 X 



Denote by Ac the map 0* i— >■ from the space C^(]R" ^ \ {0}) into 
the space by the preceding formula. By our identifications, this 
extends to an isometry Ac from Wu onto the Hilbert space 7r_M ~ tt^. 
This map exists for < m < 1. The G action on the image tt-^, gives, 
via this isomorphism Ac, the G-action on the Hilbert space Wu- 
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We similarly have a map Ah from the space C^(M"~^ \ {0}) into 
(of corresponding functions on the group H) which extends to an 

isometry from the space W'-u' onto a-u' — The H action on 
gives an action of H on the space W'^u'- 

Let J* : Wu Wu' be the adjoint of the map J. 

Proposition 14. We have the formula for the adjoint J* of J: 

J* — o res o Ag- 

(Here res : — > ^^u' simply restricting the functions on G to 
the subgroup H (since (n — l)u — 1 = (n — 2)u', it follows that the 
restriction maps the functions in JF_„ into the functions in J-'^^,)). 

Proof. The adjoint is defined by the formula 

< 9, J*f >w^,=< JgJ >wu, 

for aU functions / G C^(R'^-^ \ {0}) and g e C^°°(M"-2 \ {0}). We 
compute the right hand side of this formula. By definition, the inner 
product of Jg and / is the integral 

/ ^T:^:i^J9{x)f{x). 

Write X e R""^ = W'^ x R as x = (|/, s). Then, by the definition of 
J, Jg{x) — Jg{y, s) — g{y) and the preceding integral is the integral 

By changing s to | y | s, the integral becomes 

\y\in%u-My) ^ (l + s2)(n-lW2/(^/' I ^ I 

By the definition of the W'u' inner product, this is < g,h >w' where 
h is the function in y defined by 

f ds 

which is the integral over the variable s in the preceding equality. 
We have proved that < Jg, f >=< g, h >. 

By the definition of the adjoint, this means that h — J*f. We have 
therefore 

f ds 
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We now compute ^j/J*(/)(x') at a point x e M""^. By the definition 
of Ah, this is the integral 



y 



\{n-2)u' 



The above integral formula for J* shows that AHJ*{f){x) is the integral 

Changing the variable s back to ||j and substituting it in this integral 
(and using the fact that (n — 2)u' = {n — l)u — 1), we get 

The latter is simply AG{f){x) for x E M""^. By taking / to be Aq^(P, 
we get, for all x G M"~^, and all (p which are Ag - images of compactly 
suported smooth function on R"~^ \ {0}, that 

AHrAc'mx) = (j){x) = res{(j)){x). 

This proves the Proposition. □ 

Corollary 1. The isometric map J : W'u' — >■ Wu is equivariant for the 
action of H on both sides. 

Proof. It is enough to prove that the adjoint J* of J is equivariant for H 
action. It follows from the Proposition, that J* — Aff^oresoAo. All the 
maps Ah, Ag and res are if-equivariant. The Corollary follows. □ 

Corollary 2. The map 7r_„ — > a^u' from the module ofK-finite vectors 
in TT-u onto the corresponding vectors in a^u' is continuous. 

Proof. The map res is, by the Proposition, essentially the adjoint J* of 
the map J. But J is an isomctry which means that J* is a continuous 
projection. Therefore, res is continuous. □ 

Theorem 15. Let < -u < 1 andu' ~ ^""„-2~^ ■ -^^^ complementary 
series representation 5v of S0{n — 1, 1) occurs discretely in the restric- 
tion of of the complementary series representation tt^ of SO{n, 1). 

Proof. Wc have the i7-equivariant isometry from W^^ into Wu- Th 
former space is by construction, isometric to au' and the latter, to tt^. 
Therefore, the Theorem follows. □ 
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4. Ramified Complementary Series 



In this section, we obtain an embedding of the ramified complemen- 
tary series representation of H = SO{n — 1,1) in the ramified comple- 
mentary series representation of G = S0{n — 1, 1). The method follows 
that for the unramified case, except that in the unramified case, the G 
-invariant pairing on the complementary series was directly proved to 
be positive definite, using Theorem [TOl However, in the ramified case, 
this seems more difficult to prove. But, the positive definitenesss is 
actually known by [Ba-B] : given this positive definiteness, the proof 
proceeds as in the unramified case. Nevertheless, we give complete de- 
tails, since we will have to deal not with scalar valued operators, but 
with matrix valued ones, and some complications arise. 

4.1. A Model for Ramified Complementary Series. Let 1 < i < 

[n/2] be an integer. Set W = W^^^. Denote by S{W,i) the space 
of functions on W of Schwartz class, with values in the vector space 
A* = A*pAf (recall that pM ^ C"^^ is the standard representation of 
M = SO{n — 1) twisted by a character on 0(n — 1)). 

As we saw before, G/P = K/M and the map x ^ u{x)wP G G/P 
maps W injectively onto the "big Bruhat Cell" in G/P. Let m G C. 
Denote by J^uii) the space of continuous functions on G with values 
in A* which transform acording to the formula 



for all {g,m,a,n) e G X A X M X N. 

An element in the Schwartz space S{W, i) may be extended to a section 
of the vector bundle on G/P associated to A*) (that is, an element of 
^u{i))- We denote this embedding by E(u) : S{W,i) C JF„(z). 

Lemma 16. If Re{u) > and f G J-'uii), then the integral 



converges and as a function of g E G, lies in JF_„(z). 

Proof. The boundedness of M shows that the convergence amounts to 
the convergence (for Re{u) = C) of the integral 

f dx 



the latter integral converges because of the assumption C > 0. 



figman) = A\m){f{g))pp{ap'+'^\ 





□ 
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Given / G S(W,i) C J^u{i)-, we can then form the inner product 
(in the vector space A*) of the vectors f{{u{y)w) and Ii{u){f ){u{y)w). 
By an abuse of notation, we write f{y) = f{u{y)w). Then the inner 
product is the function x i— >< f{y), Ii{u){f){y) >. Since f{y) is in the 
Schwartz space and Ii{u)f{y) is bounded on W, it follows that this 
inner product function is integrable over y and thus we get a pairing 



< f,Ii{u)f >= dy< 
Jw 



f{y),h{u){f){y)> . 



This is a G-invariant bilinear form on S{W, i). We now use a Theorem 
of [Ba-B] to obtain that if < m < then this pairing is a positive 
definite inner product on S(W,i). 

Recall from Lemma H] that if a; G M"^^, then the Bruhat decomposi- 
tion of an element of the form 



2x 

wu(x)w = u(- —)wm{x)d{a)u{x) 

X r 



with m(x) = 1 — ^ M being the reflection about the orthogonal 

\x\ 

I 1 2 

complement of x, and a = defining the diagonal matrix d{a). 
We now prove a Lemma analogous to Lemma [TTl 

Lemma 17. // Re{u) > then the formula for the inner product is 
given, for f ^ Tu and y G M""^, by the formula 

{I{u)f){wu{y))= - — ,,-^.,^. m{x){f{y + x)). 

In this equation, the function f has values in the representation space 
A* of the group M andm{x), being an element of M, acts on the vector 
f{u{y + x)w), which, by an abuse of notation, is written f{y + x). 



Proof. The intertwining operator is 



dx 



h{u)f{u{y)w) = I y—j^;^^^j^f{u{y)wu{x)w) 



X 



We now use Lemma S] to conclude that 

I,iu)fiy) = [ dx {m{x)){f{y + ^)). 



The Lemma now follows from Lemma [5l since 



mi-. — nr) = mix) 
X r 



the two reflections are the same since the orthogonal complements are 
the same. □ 
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In the following we denote by mi{x) the linear operator A*(m(x)) 
acting on the representation space A* of M. 

We now use Lemma [8] to conclude the following. Given f E J-'u, the 
pairing of J-'uij) and is given by 

L{< f{y), Ii{u)f{y) >aO = / dy< f{y), Ii{u)f{y) >a> • 

Using the formula for the intertwining operator in Lemma [T71 we 
then have 

Lemma 18. The inner product of f with itself in J-'u{i) is given by 
< fj >:F^{i)= [ dy < f{y), [ /^^ mi{x){f{y+x)) > . 



-1 X 



If we now assume that / G S(W,i) C J-'u{i), then we see that the 
double integral in the foregoing Lemma can be interchanged. The in- 
tegral of a product of Schwartz class functions is the integral of their 
Fourier transforms. The Fourier transform of a translate of a function 
by a vector is the multiple of the transform of the function by a charac- 
ter. Using these observations (the Fourier transform of f{y) is denoted 
f{y)), we have, for (p G S{W, i), the formula 

< (j),Ii{u)(f) >s(W,i) = 

dec f ^ ^ 

dy < (/)(?/), mi > e-^'y-"". 



|(n-l)(l-«) 



-1 X 



Definition 1. Set W = M""^ and S(W,i) the space of A* valued 
Schwartz class functions on W, as before. We denote by < 0, >Ty„, 
for G S{W,i), the sesquilinear form 

f dx f 

< (P,(P>w^ii)= / ] — un-i)(i-u) / < (Piy)^^ii^)iM >A' e-^'^-^ 



w 



Since the map E{u){(f)) G J-'u{i) defined in section (4.1) is an 
injection, it follows from the result of [Ba-B] quoted earlier that the 
above sesquilinear form is positive definite, provided u is real and < 
M < 1 — Denote by Wu{i) the completion of the Schwartz space 
S{W,i) with respect to this inner product. We have then an isometry 
T^uii) Wu{i), which, on Schwartz class functions ~ E{u){(p) in 
TTuii), is given by the Fourier transform i-^ 0. 
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Definition 2. We define analogously, the subspace W = M"~^ and if 
< u' < 1 — we get a metric on the the Schwartz space S(W', i) 
given by 

<(pA>w',(i)= j 1 — u^^uTz;^ [ dy <(j){y),mi{x){(j){y)> e-^'^'-y. 



I ^ |(n-2)(l-u') , 

The completion is denoted Wl^,{i). This is isometric to the representa- 
tion oiH = SO{n - 2, 1). 

4.2. Embedding of the Ramified Complementeiry Series. 

Notation 4. We have the subgroup H = S0{n-1, 1) C G = SO{n, 1); 
the embedding is so that the intersection = M (IH is the compact 
part of the Levi subgroup of the minimal parabolic subgroup P H H 
of H; we have the representation pM^ and we have a natural inclusion 
pMDH C pM induced by the inclusion of the Lie algebra 1^ of if in the Lie 
algebra g of G. Consequently, we have the inclusion A*Pmh ^ A*pM 
and functions with values in the subspace A*Pmh may be viewed as 
functions with values in the space A*pM- 

Wc define a map J : W'u'{i) into A* valued continuous functions on 
W = M"-^ xR, by the formula 

Jmy,t) = <P{y), 

for all {y, t) E M""^ xR = W'xR = W. Using the convention of 
the preceding paragraph, we have viewed 4> '■ W A*pMn// as taking 
values in the larger vector space A'pM- 

Lemma 19. Suppose that < u < 1 — :^ and that (n — 2) (!—«') = 

(n — 1)(1 — m). The map J extends to an isometry from W'u'{i) into 
the Hubert space W„(i). 

Proof. It is enough to check that for in the Schwartz space Wu'{i), 
the function J{4>) has finite Wu{i) norm and that its norm is the norm 
of as an element of W'u'{i)- 

We then compute the latter norm. Set A = (rz — 1)(1 — -u). By as- 
sumption, X = {n — 2)(1 — u'). Recall that mi{x,t) is the i-th exterior 
power of the element m{x,t) € M and is viewed as an endomorphism 
of A*pM- We have 

Ti Ti f dx dt 

< J(p,J(p >Wu{i)^ I 

Jw'> 



/ 



X p 

dy ds < 4>{y),mi{x,t)4>{y) > e-^-^/e-^--*. 
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We have used here the fact that J(f){y,s) = (f){y). The assumption 
that is of Schwartz class imphes that all these integrals can be inter- 
changed, and hence we may write 

< J0, >= / ^ < M,{xmy) e-^-•^ 

JW'xW' I ^ I 

where Mi{x) is the linear operator on the finite dimensional vector 
space A* given by the integral 

where Tt denotes the Fourier transform with respect to the real variable 
t. 

Recall that mi{x,t) is the i-th exterior power of the matrix 1 — 2-^ 
where a = (x, t) is the row matrix of size n — 1. Therefore, the product 
^i{x,t) = (I a; p +t'^ymi{x,t) is a homogeneous polynomial of degree 
2i on the entries of the row matrix [x, t). 

We may write 
(7) 

M,{xnX/2+i) = r^v'/'-'-e-^ [ds{[ $.(x,t)e-^^-0- 
Jo V Jr (v I ^ 1 +t ) 

Changing v to v{ \ x l"^ +^^), we obtain 

M.ix)^-—^—— r^^V2+^e-N^ [ds{[ rft$.(a;,t)e-''*^-^-*). 
r(A/2 + z)Jo V Jr Jw. 

Set 

Changing i to i this integral, we obtain that 

Jr 

Thus Gi{v, s) is essentially the Fourier transform of some function eval- 
uated at s/-\/v. 

The Fourier transform of e~*^ is itself and the Fourier transform of 

the product of e~* with a power P of t is the j-th derivative of e~* . 
The jth derivative is of the form e^* Pjit) where Pj is a polynomial of 
degree j. Since ^i{x,t) is a homogeneous polynomial of degree 2i, it 
follows that for some polynomail P* of degree i, we have 
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Then we get 



V 



(after changing s to s^/v) 



= ^r(A/2)c, 

where C is some bounded number. 

Therefore, in the triple integral defining Mj(x) in the equation ([7]), 
we see that the integrand is an integrable function in t, s, v; therefore 
its integral over t is an integrable function in s and v; the integral 
over t gives an integrable function over s whose integral is (by Fourier 
inversion), the value of the function at t = 0. We therefore get 

r(A/2 + «)Jo V \x\^ 

(the last equality follows by changing v to v/{\ x |)^). 

Plugging in this value for Mj(x) in the expression for < J0, Jcf) > 
above, we obtain 

<J(p,J(p>= [ [ dy < 0(?/),mi(x,O)0(?/) > e~2*^-^. 

Jw I ^ I Jw 

By the definition of norm on W'u'H) this is < 0, >w' U), since by 
assumption, X = {n — 1)(1 — u) = {n — 2)(1 — u'). 

Therefore, J(f) has finite norm and its norm is equal to the norm of 
(j). This proves the Lemma. 

□ 

Notation 5. Denote by Ac = Ag{u) the composite of the two maps: 
(j) ^ (j) from S{W,i) into E{u){S{W,i) C TTu{i) and the intertwining 
map Ii{u) : E{u){S{W,i)) T-u{i)- If < m < 1 - then Ii{u) 
is an isomorphism, and we get a metric on the image of /»(«) whose 
completion is denoted 7r_u(i); the map Ii{u) yields an isometry Ii{u) : 

The composite of the Fourier transform with this isometry Ii{u) gives 
an isometry 

Ag = Aciu) : W^(i) 

The G-action on Wu{i) is borrowed from TTu{i) via the isometry induced 
by the Fourier transform. Therefore, the map Ac is a G-equivariant 
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map. 

We have similarly the isometry 

Ah = Ah{u) : W'u'i^ ^ ^-u'{i), 
Similarly, the map Ah is an H -equivariant isometry. 

Notation 6. Given that J is an isometry from W'u'{i) into Wu{i)., 
denote by J* its adjoint; J* is the orthogonal projection from Wu{i) 
onto W'u'{i)- 

The functions in jF_u(z) have values in the representation A'pM of 
the group M = SO (n — 1 ) and have the tranformation property 

figmdia)n) = A\m)ifig))ppidia))-^'-^\ 

Here, if d{a) is the diagonal matrix in SO{n, 1) corresponding to a G M* 
with a > 0, then pp{d{a)) = a"~^. Consequently, 

pp(rf(a))-(i-") = a("-i)(i-") = a("'2){i-«'). 

Denote by A*pA/^ the representation of MClH = S0{n — 2) on the i-th 
exterior power of its representation on pMnH, and view A^pM/^ as an 
M n if-equivariant quotient of the representation A*p m of M. There is 
no conflict with the previous notation where A''p mhh was viewed as a 
subspace of A*pM, since it ocurs with multiplicity one in the represen- 
tation A^pAf, of M nH. 

Consequently, we have a restriction map (actually, restriction fol- 
lowed by the above quotient map) 

res : J^-u{i) J^-u'{i), 

which is if-equivariant. 

We now prove an analogue of Proposition [141 for the ramified case. 

Proposition 20. Suppose that < m < 1 — and define u' by 
{n — \){\ — u) = {n — 2)(1 — u'). We have the equality 

Ah o J* = res o Ac, 

with Ag = Ag{u) and Ah = Ah{u'). 

Proof. Let 0, ^/^ G S{W,i). Since Ac is an isometry, we have the equal- 
ity 
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Now Ag{4>) = I{u) o 0. If <, >jc-^xjc-_„ denotes the pairing between 
J-u{i) and then we get 

By Lemma [U this is equal to 

dx < (f){x), {Ag^){x) >am 

w 

which is the inner product of the functions involved. We have 
therefore proved 

(8) < 0, (^gV^) >l^(w) ■ 

Similarly, we have for the subgroup H and 0, G W'u'{i), the formula 

(9) < 0, -0 >iyvW"^ (-^G^) >L2(w^')' 

where 0i is the Fourier transform for the subspace W. From equa- 
tion® we get for G W'(i), 

< J(l),1p >Wu{i) = < ■J<P,A.Glp >L'^W) = < (t),AGlp >L2(W) = 

dx dt < 0(x), {AG'ip){x,t) > . 



'W'xR 

The Fourier transform on W' x M is the Fourier transform on W' fol- 
lowed by the Fourier transform in the variable t G M. Moreover, the 
integral of the Fourier transform in t is the value of the function at 
(Fourier inversion). This shows that the last integral is equal to 



<J0, '0>= / dx < (j){x),AG'ipi{x,0) >=< (j)i,res o Ag'^P >L'2{W') ■ 
Jw 

On the other hand, from the definition of the adjoint J* and equation 
Q, we get 

< J(f),llj >Wu{i) = < <P,J*'^ >Wl,{i)> = < 01,^// oj*^ >L^W') ■ 

The last two equations show that 

Ah o J*4' = res o AgiP, 

for all Schwartz functions ijj on W. Therefore, the Proposition follows. 

□ 

Corollary 3. The map J : W'u'{i) Wu{i) is equivariant for the 
action of H on both sides. 
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Proof. By the Proposition [201 the map J* satisfies Ah° J* = res o Ac- 
All the maps Ah, Aq and res are equivariant for H . The corollary 
follows. □ 

Corollary 4. The restriction map res : S{W-u{i)) S(W'_^i{i)) is 
continuous for the G and H invariant metrics on the Schwartz spaces 
S{W.u,i) andS{W'_^,,i). 

Proof. The map res = Ah o J* o Aq^; the maps Ac, Ah are isome- 
tries and J* is an orthogonal projection (adjoint of an isometry J). 
Consequently, res is also an orthogonal projection and is in particular, 
continuous. □ 

Theorem 21. The representation on the Hilbert space (T„'(i) occurs 
discretely in the Hilbert space 7Tu{i) if < m < 1 — 

Proof. We have in fact (Corollary H]) that the map 

res : 7r„„ a_u> 
is continuous for the norms on the representations involved. 

□ 
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5. The Main Theorem and Corollaries 

Theorem 22. Let n > 3 H = SO{n - 1, 1) and G = SO{n, 1). Let 
i be an integer with < i < [n/2] — 1 where [x] denotes the integral 
part of X. Let < u < 1 — Then the complementary series 

representation 

TTuii) = IndfiA'pM ® pp(a)"), 
contains discretely, the complementary series representation 

where u' = , Further, as u tends to 1 — ^ , u' tends to 1 — ^ . 

Proof. We have aheady proved this (see Theorem [2T1) except for the 
last statement, which is trivial to prove. □ 

Corollary 5. Let i < [n/2] — 1. Then the cohomological representa- 
tion Ai{n) of SO{n, 1) of degree i restricted to SO{n — 1, 1) contains 
discretely, the cohomological representation Ai{n — 1). 

The discrete series representation Ai{2i) of S0{2i, 1) is contained 
discretely in the representation Ai. 

Proof. To prove the first part, we use Theorem [221 and let u tend to the 
limit 1 — This gives: Ai{n) restricted to SO{n — 1,1), contains 

Ai{n — 1) discretely. The rest of the corollary follows by induction. □ 

Corollary 6. Suppose that for all n, the tempered cohomological 
representations Ai{n) (i.e. i = [n/2]) are not limits of complementary 
series in the automorphic dual of SO{n,l). Then, for any n, all 

nontempered cohomological representations of SO{n,l) are isolated in 
the automorphic dual. 
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